Prediction-based provisioning planning for cloud environments

ABSTRACT

Various embodiments predict performance of a system including a plurality of server tiers. In one embodiment, a first set of performance information is collected for a base allocation of computing resources across multiple server tiers in the plurality of sever tiers for a set of workloads. A set of experimental allocations of the computing resources is generated on a tier-by-tier basis. Each of the set of experimental allocations varies the computing resources allocated by the base allocation for a single server tier of the multiple server tiers. A second set of performance information associated with the single server tier for each of the set of experimental allocations is collected for a plurality of workloads. At least one performance characteristic of at least one candidate allocation of computing resources across the multiple server tiers is predicted for a given workload based on the first and second sets of performance information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is continuation of and claims priority from U.S. patentapplication Ser. No. 13/627,672 filed on Sep. 26, 2012, now ______, thedisclosure of which is hereby incorporated by reference in its entirety.

BACKGROUND

The present invention generally relates to cloud computing environments,and more particularly relates to provisioning resources within a cloudcomputing environment.

In general, cloud computing refers to Internet-based computing whereshared resources, software, and information are provided to users ofcomputer systems and other electronic devices (e.g., mobile phones) ondemand. Adoption of cloud computing has been aided by the widespreadadoption of virtualization, which is the creation of a virtual (ratherthan actual) version of something, e.g., an operating system, a server,a storage device, network resources, etc. Cloud computing provides aconsumption and delivery model for information technology (IT) servicesbased on the Internet and involves over-the-Internet provisioning ofdynamically scalable and usually virtualized resources.

Cloud computing is facilitated by ease-of-access to remote computingwebsites (via the Internet) and frequently takes the form of web-basedtools or applications that a cloud consumer can access and use through aweb browser, as if the tools or applications were a local programinstalled on a computer system of the cloud consumer. Commercial cloudimplementations are generally expected to meet quality of service (QoS)requirements of consumers and typically include service level agreements(SLAs). Cloud consumers avoid capital expenditures by renting usage froma cloud vendor (i.e., a third-party provider). In a typical cloudimplementation, cloud consumers consume resources as a service and payonly for resources used.

BRIEF SUMMARY

In one embodiment, a computer program product for predicting performanceof a system comprising a plurality of server tiers is disclosed. Thecomputer program product comprises a storage medium readable by aprocessing circuit and storing instructions for execution by theprocessing circuit for performing a method. The method comprisescollecting a first set of performance information for a base allocationof computing resources across multiple server tiers in the plurality ofsever tiers for a set of workloads. A set of experimental allocations ofthe computing resources is generated on a tier-by-tier basis. Each ofthe set of experimental allocations varies the computing resourcesallocated by the base allocation for a single server tier of themultiple server tiers. A second set of performance informationassociated with the single server tier for each of the set ofexperimental allocations is collected for a plurality of workloads. Atleast one performance characteristic of at least one candidateallocation of computing resources across the multiple server tiers ispredicted for a given workload based on the first set of performanceinformation and the second set of performance information.

In another embodiment, an information processing system for predictingperformance of a system comprising a plurality of server tiers isdisclosed. The information processing system comprises a memory and aprocessor communicatively coupled to the memory. A provisioning manageris communicatively coupled to the memory and the processor. Theprovisioning manger is configured to perform a method. The methodcomprises collecting a first set of performance information for a baseallocation of computing resources across multiple server tiers in theplurality of sever tiers for a set of workloads. A set of experimentalallocations of the computing resources is generated on a tier-by-tierbasis. Each of the set of experimental allocations varies the computingresources allocated by the base allocation for a single server tier ofthe multiple server tiers. A second set of performance informationassociated with the single server tier for each of the set ofexperimental allocations is collected for a plurality of workloads. Atleast one performance characteristic of at least one candidateallocation of computing resources across the multiple server tiers ispredicted for a given workload based on the first set of performanceinformation and the second set of performance information.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, and which together with the detailed description below areincorporated in and form part of the specification, serve to furtherillustrate various embodiments and to explain various principles andadvantages all in accordance with the present invention, in which:

FIG. 1 is a block diagram illustrating one example of an operatingenvironment according to one embodiment of the present invention;

FIG. 2 is a block diagram illustrating a detailed view of a provisioningmanager according to one embodiment of the present invention;

FIG. 3 shows one example of a per-tier model training process accordingto one embodiment of the present invention;

FIG. 4 shows an overall view of a prediction process for predicting theperformance of a base deployment and target deployments according to oneembodiment of the present invention;

FIG. 5 is an operational flow diagram illustrating one example of anoverall process for performing prediction-based provisioning planning ina cloud computing environment according to one embodiment of the presentinvention;

FIG. 6 is an operational flow diagram illustrating one example of aprocess for predicting the performance of candidate provisioning plansaccording to one embodiment of the present invention;

FIG. 7 is an operational flow diagram illustrating one example of aprocess for performing automatic provisioning experiments on a currentlydeployed cloud application according to one embodiment of the presentinvention;

FIG. 8 illustrates one example of a cloud computing node according toone embodiment of the present invention;

FIG. 9 illustrates one example of a cloud computing environmentaccording to one example of the present invention; and

FIG. 10 illustrates abstraction model layers according to one example ofthe present invention.

DETAILED DESCRIPTION

Deploying a multi-tier web application to meet a certain performancegoal with minimum virtual instance renting cost is often the goal ofmany Infrastructure-as-a-Service (IaaS) users. However, achieving thisgoal can be very difficult to achieve for several reasons. First, atypical IaaS environments offer a variety of virtual server instanceswith different performance capacities and rental rates. Such instancesare often marked with a high level description of theirhardware/software configuration (e.g. 1 or 2 virtual CPUs) which offerslittle information regarding their performance for a particularapplication.

Second, multi-tier web applications often leverage clusters at differenttiers to offer features such as load balance, scalability, and faulttolerance. The configuration of clusters (e.g., the number of membernodes, how workloads are distributed among member nodes, etc.) has adirect impact on application performance. However, the relation betweencluster configuration and performance is application-dependent, andoften not clear to cloud users.

To meet a given performance goal, users often over-provision amulti-tier web application by renting high-end virtual server instancesand employing large clusters. Over-provisioning introduces high instancerenting cost, which can make cloud deployment a less desirable optioncompared with traditional deployment options. Unfortunately, manuallyexperimenting with different provisioning plans is often impracticalgiven the huge space of candidate provisioning plans.

Therefore, one or more embodiments of the present invention provideprediction-based provisioning planning. Prediction-based provisioningplanning identifies the most cost-effective provisioning plan for agiven performance goal by searching the space of candidate plans withperformance prediction. One or more mechanisms are provided thatefficiently learn performance traits of applications, virtual machines,and clusters to build models to predict the performance for an arbitraryprovisioning plan. Historical performance monitoring data and datacollected from a small set of automatic experiments are utilized tobuild a composite performance prediction model. This compositeperformance prediction module takes as input application workloads,types of virtual server instances, and cluster configuration, andoutputs predicted performance.

Operating Environment

FIG. 1 shows one example of an operating environment 100 forprovisioning resources in a cloud computing environment for multi-tiercloud applications. It should be noted that although the followingdiscussion is directed to a cloud computing environment variousembodiment are not limited to such environment and are application tonon-cloud computing environments as well. In particular, FIG. 1 showsone or more client/user systems 102 communicatively coupled to one ormore cloud computing environments 104 via a public network 106 such asthe Internet. The user systems 102 can include, for example, informationprocessing systems such as desktop computers, laptop computers, servers,wireless devices (e.g., mobile phones, tablets, personal digitalassistants, etc.), and the like.

The user systems 102 access the cloud computing environment 106 via oneor more interfaces (not shown) such as a web browser, application, etc.to utilize resources provided by the environment 104. For example, FIG.1 shows a plurality of resources such as applications 108 and computingresources 110 available within the cloud computing environment 104.Computing resources 110 include but are not limited to, processing,storage, networking, and other fundamental computing resources.Resources 108, 110 are provided by and/or are hosted on a plurality ofphysical information processing systems 112, 114, 116 and/or a pluralityof virtual machines 118, 120 being executed by physical systems 114,116. A plurality of physical systems 112, 114, 116, virtual machines120, 122, or a combination thereof grouped together for providing aresource(s) is referred to as a “cluster” 124.

In one example, a cloud user (via a user system 102) utilizes the cloudenvironment 104 to deploy a multi-tier web application. In this example,a multi-tier web application is an application (software designed toenable a user to perform a given task) accessible over a network whosepresentation, logic (application processing), and data storage processesare performed at physically separate tiers. For example, thepresentation processes can be performed on a web server tier; theapplication processing can be performed on an application server tier;and the data storage processes can be performed on a database servertier. Each of the web server, application server, and database servertiers can be comprised of one or more of the information processingsystems 114, 116 and/or VMs 120, 122 in the cloud environment 104.

The cloud computing environment 104 further comprises one or moreinformation processing systems 126 that comprise a provisioning manager128. It should be noted that the information processing system 126 isnot required to reside within the cloud environment 106. Theprovisioning manager 128 provisions resources in the cloud environment106 to cloud users. In one embodiment, the provisioning manager 128collects a first set of performance information for a base allocation ofcomputing resources across multiple server tiers in the plurality ofsever tiers for a set of workloads. The provisioning manager 128 alsogenerates a set of experimental allocations of the computing resourcesis generated on a tier-by-tier basis. Each of the set of experimentalallocations varies the computing resources allocated by the baseallocation for a single server tier of the multiple server tiers. Asecond set of performance information associated with the single servertier for each of the set of experimental allocations is collected by theprovisioning manager 128 for a plurality of workloads. At least oneperformance characteristic of at least one candidate allocation ofcomputing resources across the multiple server tiers is predicted by theprovisioning manager 128 for a given workload based on the first andsecond sets of performance information.

The provisioning manager 128, in one embodiment, comprises anapplication monitor 202, a model trainer 204, an experiment manager 206,a predictor 208, and a provisioning plan selector 210, as shown in FIG.2. The application monitor 202 monitors and records applicationworkloads and the corresponding performance. The model trainer 204trains cross-tier performance models 212 based on the collectedworkloads and performance data. The model trainer 204 also trainsper-tier performance models 214 based on workloads and performance datacollected by the application monitor 202 during automatic experimentsperformed by the experiment manager 206. The experiment manager 206replicates the multi-tier application, which was deployed for a clouduser, for a set of automatic experiments. These automatic experimentsdeploy the application with different provisioning plans and measure thecorresponding performance with different workloads. The automaticexperiments learn the performance characteristics of differentdeployment options (e.g., virtual machine types and the number ofvirtual machines in a cluster). The predictor 208 analyzes a pluralityof candidate provisioning plans and predicts the correspondingperformance (for the user specified workload range) using both thecross-tier and per-tier performance models 212, 214. The provisioningplan selector 210 selects the candidate provisioning plan that meets theuser-specified performance goal and has the lowest virtual machineinstance renting cost. This selected plan is utilized as the suggestdeployment for the cloud user. The provisioning manager 128 and itscomponents are discussed in greater detail below.

Predictive-Based Provisioning

The following is a more detailed discussion regarding theprediction-based provisioning planning performed by the provisioningmanager 128. Throughout this discussion interactive cloud applicationssuch as web applications are used as one example of the targetedapplications. Such applications are request-driven and one request maybe served by multiple components at different tiers (e.g., web servers,application servers and database servers). The performancegoal/characteristic, such as (but not limited to) response time,requested by the cloud user is used to measure the performance ofapplications. Another performance goal/characteristic is request rate(throughput), which is used to measure the workloads on applications. Itshould be noted that other metrics can be utilized to measure theperformance and workloads as well. The term “deployment” as used hereinrefers to the choice of virtual machine type and cluster configuration(the number of member nodes).

The provisioning manager 128, in one embodiment, identifies the mostcost-effective provisioning plan for a given performance goal bysearching the space of candidate plans with performance prediction. Theprovisioning manager 128 efficiently learns performance traits ofapplications 108, virtual machines 120, 122, and clusters 124 to buildmodels 212, 214 for predicting the performance for an arbitraryprovisioning plan. The provisioning manager 128 utilizes historicalperformance monitoring data and data collected from a small set ofautomatic experiments to build a composite performance prediction model.This performance prediction model that takes as input applicationworkloads, types of virtual server instances, and cluster configurationand outputs predicted performance.

The provisioning manager 128 avoids exhaustively performing experimentson all candidate deployments to build a performance prediction model byusing a two-step performance prediction procedure. For example, insteadof directly predicting the performance of an arbitrary (target)deployment (also referred to here as “candidate allocation”), theprovisioning manager 128 first predicts the performance on a known(base) deployment (also referred to herein as “base allocation”) andthen predicts the performance differences between the target deploymentand the base deployment. The provisioning manager 128 combines thepredicted base performance and the predicted performance changes toobtain the performance on the target deployment.

To achieve efficiency, the provisioning manager 128 predicts theperformance change (delta) based on the deployment difference betweenthe base deployment and the target deployment within each tier of themulti-tier application, rather than predict the overall performancechanges holistically across multiple tiers. This avoids the need toexhaustively explore all deployments that represent combinations ofdeployment changes across tiers, since the provisioning manager 128considers each tier independently. For example, suppose an applicationincludes 3 tiers and each tier has 10 possible forms. An exhaustivesearch would explore all 10³=1000 deployments to train a traditionalperformance prediction model, while the provisioning manager 128 wouldonly test 3*10=30 deployments to obtain the two-step performanceprediction model. The provisioning manager 128 also applies amultiplicative-delta learning technique (in capturing performancechanges introduced by different sizes of a tier) to further reduce thenumber of required experiments for model training. In addition, theprovisioning manager 128 addresses cross-tier workload characteristicschanges that violate the inter-tier independence of the performancemodel.

In one embodiment, the planning performed by the provisioning manager128 comprises a prediction method(s), a capturing method(s), and aplanning method(s). The prediction method takes workloads and deploymentas input, and outputs the predicted application performance. Thecapturing method captures the changes of perceived workloads acrossdifferent deployments. The planning method explores all candidateprovisioning plans and outputs the optimal one.

With respect to prediction, the predictor 208 of the provision manager128 predicts the response time for a given workload on anover-provisioned deployment (also referred to as the base deployment).The predictor 208 then modifies the predicted response time consideringchanges introduced by the difference between the over-provisioneddeployment and the actual targeted deployment. Two performance modelsare utilized to accomplish this task: a cross-tier performance model 212and a per-tier performance model 214. The cross-tier performance model212 captures the relation between workload and response time for thebase deployment. The per-tier performance model 214 captures therelation between deployment changes (to the base deployment) andcorresponding changes of the response time.

A cross-tier model has the following form,

Θ_(c)(w)→r  (1)

where w is the workload and r is the average response time of requests.The cross-tier model 212 takes workload (actual and/or observed at thevarious server tiers in the cloud) as input and outputs the responsetime on the base deployment. It should be noted that even though averageresponse time is used to describe the techniques, one or more embodimentalso support the prediction of quantile response time (e.g., 90thpercent response time of requests). In one embodiment, the model trainer204 trains the cross-tier model 212 using one or more trainingmechanisms. One example of a training mechanism is Kernel regression,which is a non-parametric technique that does not specify a certainrelation (e.g., linear relation) between w and r, but produces anonlinear relation between w and r that best fits the observedperformance data. This flexibility is important as the actual relationbetween w and r may vary at different workload levels, or acrossdifferent applications.

A per-tier model 214 has the form of,

Θ_(p) ^(t)(w,v,c)→r _(Δ)  (2)

where t denotes the object tier, v is the virtual machine type, c is thecluster size, i.e. the number of member nodes, and r_(Δ) is the changeof response time compared with the base deployment. The per-tier model214 is a set of models where each model is trained for a particulartier. Each per-tier model takes the workload and the type and the numberof virtual machine used at the object tier as input and outputs thechanges of response time introduced by this tier over that of the basedeployment. Similar to the cross-tier model 212, the pier-tier model 214is trained by the model trainer 204 using one or more trainingmechanisms such as (but not limited to) Kernel regression.

To predict the response time for a target deployment and a givenworkload, the predictor 208 uses the per-tier model 214 to estimate thedifferences of response time introduced at each tier due to thedeployment differences between the target deployment and the baseddeployment. Specifically, the overall change of response time changeR_(Δ) is,

$\begin{matrix}\left. R_{\Delta}\leftarrow{\sum\limits_{\forall t}^{\;}\; {\Theta_{p}^{t}\left( {w,{v(t)},{c(t)}} \right)}} \right. & (3)\end{matrix}$

where v(t) is the virtual machine type in tier t and c(t) is the numberof virtual machines in tier t. The final predicted response time r* is,

r*←R _(Δ)+Θ_(c)(w)  (4)

where the predictor 208 applies the predicted response time changes tothe predicted response time on the base deployment.

In one embodiment, the cross-tier model 212 and the per-tier model 214are trained separately by the model trainer 204 in two steps. The modeltrainer 204 trains the cross-tier model 212 with performance monitoringdata associated with the base deployment. This data can be collectedfrom the base deployment when it serves user requests. Therefore,additional experiments are not required for data collection. In oneembodiment, the training data set includes the request rates spanningfrom light workloads to peak workloads and the corresponding averageresponse time. Various statistical tools can be used to train thecross-tier model 212. Typically, the base deployment is over-provisionedto ensure the request response time meets the performance goal. However,various embodiments are applicable to any base deployment. The basedeployment is also used as a contrast to generate training data for theper-tier model 214.

The per-tier models 214 are trained in a tier-by-tier basis based onperformance data collected on a series of automatic experimentsperformed by the experiment manager 206. For example, the experimentmanager 206 creates a duplicate of the base deployment. This duplicateis referred to as the background deployment. For a per-tier model ontier t, the experiment manager 206 varies the configuration of tier t onthe background deployment by changing the virtual machine type and thenumber of virtual machines. The experiment manager 206 leaves theconfiguration of other tiers unchanged (same as the configuration in thebase deployment). This leads to mn different background deploymentswhere m is the total number of virtual machine types and n is themaximum number of virtual machines in tier t. For each resultingbackground deployment (with virtual machine type v(t) and virtualmachine number c(t) in tier t), the experiment manager 206 introducesdifferent levels of workloads (from light level to peak level just asthose in the cross-tier model training dataset) to the deployment. Theexperiment manager 206 records the differences in response time r_(Δ)between the background deployment and the base deployment for each levelof workload w. The workload, in one embodiment, is generated by workloadgeneration tools. The resulting data points (w, v(t), c(t), r_(Δ)) areused to train the per-tier model Θ_(p) ^(t). Similar to the cross-tiermodel 212, various statistical tools can be used to train the per-tiermodel 214.

One aspect of training the per-tier model 214 is capturing clusterperformance changes with different number of virtual machines. Thevirtual machine provisioning time on most cloud platforms ranges from afew minutes to 20 minutes. As a result, adding virtual machines to acluster one-by-one to capture the corresponding performance changes cantake substantial time, especially for large clusters with many membernodes. To address this issue, one or more embodiments utilize amultiplicative-delta learning technique that selectively performsadditional experiments. For example, instead of adding virtual machinesone-by-one, the model trainer 204 doubles the virtual machinesincremental number if the per-tier model gives good prediction on theperformance of the current cluster. If the prediction accuracy drops atcertain point, the model trainer 204 reduces the instance incrementalnumber by half. The procedure finishes until the maximum instance numberis reached. This technique is advantageous because most clustersimplement a load-balance scheme among their member instances. As aresult, the performance curve can be learned with relatively smallamount of training data. Even if the cluster implements a complicatedworkload assignment scheme, the technique can degenerate to the originalcluster performance learning procedure which intensively collectsperformance data points with many different size settings.

A detailed example will now be given illustrating one example ofprediction-based provisioning planning in a cloud computing environment.In this example, a web application is deployed in a cloud environmentsuch as International Business Machine's Smart Cloud Enterprise (SCE).SCE provides 9 different types of pre-configured virtual machineinstances. The configuration is defined in terms of the number ofvirtual CPUs, the size of virtual machine memory and the size of localstorage. Different types of VMs are also associated with differenthourly (renting) rate.

In this example, a user wants to deploy a web application comprising ofthree tiers, the web server tier, the application server tier and adatabase tier. To deploy the web application, the user needs to decidethe deployment plan for each tier. For example, the user needs todetermine what types of VM instances to use at one tier and how many VMinstances to use at one tier. In this example, it is assumed that onetier can at most utilize N=20 VM instances. In addition, the user alsohas a performance requirement of achieving an average request responsetime (measured in a 10-second time window) less than 2 seconds, as longas the incoming requests rate is below a certain level, e.g., 500requests per second. The overall deployment goal, in this example, is toachieve this performance goal with minimum instance renting cost.

The provisioning manager 128 builds a performance model that producesaccurate performance prediction for different deployments (versus asingle-deployment prediction model). First, the model trainer 204 trainsa regression-based performance model on an over-provisioned deploymentreferred to as the base deployment. In this example, such anover-provisioned deployment comprises Platinum virtual machines (64-bitVM with 16 virtual CPUs and 16 GB memory) and each tier has 20 such VMs.The training process includes feeding the base deployment with differentlevels of workloads and measuring the corresponding performance. Theresulting performance data (average response time) and workloads arethen used to train the performance model, which is a cross-tier model212 that can predict the average response time for a certain workload onthe base deployment.

The model trainer 204 also trains a set of models that captures theperformance changes introduced by using different VM types and differentnumber of VMs at each tier. This process is performed on a tier-by-tierbasis with an outer loop and an inner loop. The outer loop deals withone tier at a time and the inner loop captures the performance changesbrought by deployment changes at one tier. In one embodiment, the outerloop first selects the web server tier for manipulation. Within thecorresponding inner loop, the experiment manager 206 changes the typesof VMs from Platinum to 64-bit Gold (8 virtual CPUs and 16 GB memory) atthe web server tier, and measures the difference between performance onthe new deployment and also on the base deployment given differentlevels of workloads. The experiment manager 206 then reduces the numberof VMs at the web server tier one-by-one, and measures the differencebetween performance on the resulting deployment and the base deployment.Note that the VM type and number is changed at the web server tier whilethe other two tiers (the application server tier and the database tier)are left unchanged (same as those in the base deployment).

Similarly, the VM type is then changed to 64-bit Silver (4 virtual CPUsand 8 GB memory) and the number of VMs is varied at the web server tier.For each resulting deployment, the experiment manager 206 measures thedifference between performance on the new deployment and also on thebase deployment given different levels of workloads. This process isrepeated until all VM types have been tried on the web server tier. Thecollected performance difference data allows the model trainer 204 totrain a web server tier model Θ_(c)(w)→r that predicts the performancechanges introduced by deployment changes (i.e., VM type and number) atthe web server tier of the base deployment. At this point the firstround of the outer loop finishes.

FIG. 3 illustrates the per-tier training process discussed above. Inparticular, for a specific tier 302, two nested loops are used to alterthe type and the number of VMs running in the tier. In an outer loop,the type of VMs is changed to a different type in each round, as shownby boxes with different shading. In the inner loop, the number of VMs ischanged to a different number in each round. The other tiers 304, 306that are not currently used for per-tier training are kept the same asthose in the base deployment. For each of the resulting configurations(with regards to VM type and number) the corresponding performancedifference between the resulting deployment and the base deployment ismeasured. Note that the training process does not have to explore all VMtypes or VM numbers that are applicable to one tier as long as thetraining process provides sufficient data for the performance model. Forinstance, the inner loop can try 1, 2, 4, . . . , 2̂n, . . . , MAXinstead of 1, 2, 3, . . . , MAX for the VM number during the trainingprocess where MAX is maximum number of VMs that may run in the tier.

The generated data leads to an application server tier model thatpredicts the performance changes introduced by deployment changes at theapplication server tier of the base deployment. Similarly, the finalround works on the database tier and produces a database tier model thatpredicts the performance changes introduced by deployment changes at theapplication server tier of the base deployment. The resulting threetrained models are referred to as per-tier models 214. The predictor 208of the provisioning manager 128 utilizes the cross-tier performancemodel 212 generated based on the base deployment 402 and the per-tierperformance models 214 generated based on the background deployments 216to predict the performance of an arbitrary deployment, as shown in FIG.4. For example, consider a scenario where the average response time isof interest for a deployment comprising 5 Bronze VMs (2 virtual CPUs and4 GB memory) at the web server tier, 10 Silver VMs (4 virtual CPUs and 8GB memory) at the application server tier, and 20 Gold VMs (8 virtualCPUs and 16 GB memory) at the database tier when given a workload of 500requests per second. The predictor 208 uses the cross-tier model 212 topredict the average response time for the given workload (500request/second). Note that the predicted response time (e.g., the baseresponse time) is for the base deployment.

Next, the predictor 208 applies the web server tier model (a per-tiermodel 214) to predict the changes of response time contributed by thedeployment changes at the web server tier (compared with that of thebase deployment). As 5 Bronze VMs have much less processing powercompared with 20 Platinum VMs in the base deployment, the predictedresponse time change is very likely to be a positive value. Similarly,the predictor 208 applies the application server tier model and thedatabase tier model to obtain the predicted response time changes at thecorresponding tiers. Finally, the predictor 208 sums up the baseresponse time and the three predicted response time changes at differenttiers together to obtain the predicted response time for the givendeployment.

The above example makes an implicit assumption that the actual workloadsperceived at each tier do not change across different deployments. Thisassumption, however, may not hold for many cloud applications. Theperceived workload at a tier may not be the same as the workloadintroduced to the application due to prioritization, rate limitingmechanisms implemented at different tiers. For instance, an applicationmay drop certain low-priority requests when a certain tier becomesperformance bottleneck, which in turn causes the change of workload atother tiers. Even for applications without prioritization mechanisms abottleneck tier may limit the overall system throughput and introducechanges to the workload on other tiers.

Performance prediction without considering such workload changes maylead to significant prediction accuracy loss. As another example, adatabase tier of a web application configured with a single low-endvirtual machine can be a performance bottleneck when the web applicationis fed with a peak workload w_(p). As a result, the actual workloadsperceived at each tier w′ is often less than w_(p) as a certain amountof requests are queued due to database overloading. Using the data(w_(p), v, c, r_(Δ)) for training can introduce error to the per-tiermodel 214. To address this issue, one or more embodiments also utilize athroughput model Θ_(h) ^(t) for a tier t with the following form,

Θ_(h) ^(t)(w,v,c)→w′  (5)

where w′ is the actual workload perceived by all tiers. When makingperformance predictions the predictor 208 applies the throughput modelto obtain the predicted workload at each tier, and use the lowest (e.g.,smallest) predicted workload as the input of the per-tier model.Specifically, with the throughput model, the per-tier model has thefollowing form,

$\begin{matrix}\left. {\Theta_{p}^{t}\left( {{\min\limits_{\forall t}{\Theta_{h}^{t}\left( {w,{v(t)},{c(t)}} \right)}},v,c} \right)}\rightarrow r_{\Delta} \right. & (6)\end{matrix}$

where the input workload w is replaced with the actual workloadpredicted by the throughput model. Various training mechanisms such asKernel regression can be used to train the throughput model. Note thatthe data used for training the throughput model is (w, v, c, w′) and w′can be measured by counting the number of responses within a timewindow.

In addition to the above, the provisioning manager 128 also supportsrequest-mix awareness. For example, application workloads often compriserequests of different types. Requests of different types often introducedifferent processing overheads. For instance, bidding request in biddingapplications usually incur higher costs than browsing requests do asbidding often involves database transactions. As a result, even if twoworkloads have the same request rate, they may result in very differentresource consumption and performance if the composition of requests arevery different (e.g., a 100 request/second workload with 20% biddingrequests and 80% browsing requests versus another 100 request/secondworkload with 80% bidding requests and 20% browsing requests).

Performance oriented provisioning planning for application withheterogeneous per-request costs requires fine-grain definition ofworkloads with information on the composition of requests. Accordingly,the predictor 208 also considers the composition of requests, an aspectreferred to herein as “request-mix awareness”. To supportrequest-mix-aware prediction, a set of new inputs is introduced, whichdescribe the request composition of a workload. For example, theworkload w (scalar) is replaced with a vector R=r₁, r₂, . . . , r_(k)where r_(i) is the rate of requests of type i. For the brevity ofdiscussion, the overall response time for all requests is predicted.Note that the various techniques discussed above can be directly used topredict the response time for a specific type, or a set of types, ofrequests by using the corresponding response time (of the specific type,or a set of type, of requests) to train models.

Training a model that is oblivious to request composition comprisesgenerating workloads with different request rates, i.e., the model input(request rate) is a scalar. However, training a request-mix-aware modelcomprises much more performance measurement (training) data withdifferent compositions of types of requests due to the extra degrees offreedom introduced by per-request-type workloads, i.e. the model input(per-type request rate) is a vector. This can significantly increase theexperiment time and make the model training process expensive. Forexample, suppose there are 20 different types of requests and we measurerequest rates in 10 different levels (e.g., 0-100, 100-200, 200-300,etc.). An ideal set of training data would include all compositions ofper-type request rates (10²⁰ different workloads), which is notpractical. Note that even though the ideal set of data is not alwaysneeded to achieve reasonable prediction accuracy, (e.g., a 10% subset ofthe ideal training data (randomly selected) may be sufficient) a smallpercentage of such a large dataset (e.g., 10% of 10²⁰) is still notpractical to generate in man situations.

Therefore, the provisioning manager 128, in one embodiment, isconfigured to substantially reduce the needed experiment time. Forexample, the provisioning manager 128 automatically identifies acorrelation such as a cost relationship between different requests,e.g., request A and B have similar cost, or the cost of request A isabout 2 times higher than that of request B. Such cost relationshipsallow the provisioning manager 128 to map the original workload vectorinto a new workload vector with much smaller number of dimensions. Thisgreatly reduces the amount of training data needed to reflex differentworkload compositions. For the previous example, if the provisioningmanager 128 groups 20 different types of requests into 2 general types(e.g., transactional and non-transactional) the number of compositionsin the ideal training dataset is effectively reduced from 10²⁰ to 10².

The following is an illustrative example of efficiently trainingrequest-mix aware models. The provisioning manager 128 can utilizevarious methods for efficiently training request-mix aware models. In afirst method, the provisioning manager 128 removes requests with trivialoverheads from the performance model. For instance, an HTTP requestasking for a small static html file (often cached) from the web serveris removed. However, this method may not be able to substantially reducethe dimension of the model input vector as such low-cost requests oftencontribute to a very limited portion of the overall workloads (e.g.,<1%). Therefore, in another method the provisioning manager 128 clustersrequests into different groups where requests within the same group havesimilar overheads. This reduces the dimension of the model input fromthe number of request types to the number of clusters. Consider a pairof request types A and B. Requests of type A and B both cause thedatabase server to perform the same SELECT operation and the onlydifference is that the SELECT operation is executed once for A but twicefor B. Stated differently, a request of type B is approximately twotimes more expensive than a request of type A. If A and B are clusteredinto different groups with fine clustering granularities, the totalnumber of groups can be quite large as only requests with very similaroverhead are grouped together. However, if A and B are clustered intothe same group, different compositions of type A and B requests may leadto very different workloads due to overhead difference between A and B,even if the total number of requests of this general type may be thesame.

The provisioning manager 128 flexibly captures the cost relation betweendifferent request types. For example, for requests of the same group,the provisioning manager 128 captures their relative overhead with alinear system. For the previous example, the total workload introducedby requests of type A and B W_(A,B)=N_(A)+2N_(B), where N(·) is therequest number of a certain type. Formally, the provisioning manager 128linearly projects the original workload vector {right arrow over (W)}defined in a high dimensional space into a new workload vector {rightarrow over (W)}* defined in a lower dimensional space.

One difficulty in this projection process is to ensure that the new{right arrow over (W)}* can accurately represent the true workload sothat the performance model can provide good prediction. Achieving thisgoal, however, involves two challenges. The first challenge is toevaluate the quality of a projection π. Although it is possible to applyπ to get {right arrow over (W)}* from {right arrow over (W)}, andcompare the prediction accuracy of the performance model trained with{right arrow over (W)}* and that of the model trained with {right arrowover (W)}, such an approach is also prohibitively expensive given thecomputation cost of model training. The second challenge is how toefficiently explore and evaluate different projections to find anoptimal one. Brute force approaches that explore all possibleprojections are not practical due to the countless number of possibleprojections.

Therefore, with respect to evaluating the quality of a projection π, theprovisioning manager 128 evaluates the quality of a projection withoutactually training a performance model based on the projected modelinput. In this embodiment, mutual information between the projectedmodel input and the corresponding response time as the metric forevaluation, i.e., I(R, {right arrow over (W)}*) where R is the responsetime and {right arrow over (W)}* is the projected model input. Mutualinformation of two random variables is a quality that measures themutual dependence of the two random variables. Formally, the mutualinformation of two discrete random variables X and Y can be defined as,

$\begin{matrix}\begin{matrix}{{I\left( {X,Y} \right)} = {\sum\limits_{y \in Y}^{\;}\; {\sum\limits_{x \in X}^{\;}\; {{p\left( {x,y} \right)}{\log \left( \frac{p\left( {x,y} \right)}{{p(x)}{p(y)}} \right)}{I\left( {X,Y} \right)}}}}} \\{= {\sum\limits_{y \in Y}^{\;}\; {\sum\limits_{x \in X}^{\;}{{p\left( {x,y} \right)}{{\log \left( \frac{p\left( {x,y} \right)}{{p(x)}{p(y)}} \right)}.}}}}}\end{matrix} & (7)\end{matrix}$

Mutual information measures the information that X and Y share: itmeasures how much knowing one of these variables reduces uncertaintyabout the other. For example, if X and Y are independent, then knowing Xdoes not give any information about Y and vice versa, so their mutualinformation is zero. At the other end, if X and Y are identical then allinformation conveyed by X is shared with Y: knowing X determines thevalue of Y and vice versa.

Fano's inequality suggests that one can find the optimal projection π bymaximizing I(R, {right arrow over (W)}*). This result determines a lowerbound to the probability of error when estimating a discrete randomvariable R from another random variable {right arrow over (W)}* as

$\begin{matrix}{{{\Pr \left( {r \neq \hat{r}} \right)} \geq \frac{{H\left( {R{\overset{\rightarrow}{W}}^{*}} \right)} - 1}{\log \left( {R} \right)}} = {\frac{{H(R)} - {I\left( {R,{\overset{\rightarrow}{W}}^{*}} \right)} - 1}{\log \left( {R} \right)}.}} & (8)\end{matrix}$

Hence, when the mutual information between R and {right arrow over (W)}*is maximized, the lower bound on error probability is minimized.Therefore, mutual information serves as a good indicator for the qualityof projection, because the higher the mutual information is, the higherpredictability of the model built based on the projected model input is.

With respect to efficiently exploring and evaluating differentprojections to fine an optimal projection, since I(R, {right arrow over(W)}*) is used to measure the quality of a projection and the idealprojection is the one that maximizes I(R, {right arrow over (W)}*), thesearch for an ideal projection can be formulated as optimization problemdefined as follows,

$\begin{matrix}{\pi = {\arg {\max\limits_{\pi}{I\left( {R,{{\overset{\rightarrow}{W}}^{*}(\pi)}} \right)}}}} & (9)\end{matrix}$

where {right arrow over (W)}*(π) is the resulting model input generatedby using projection π. As a result, the provisioning manger 128 canperform gradient ascent on I to find the optimal projection as follows,

$\begin{matrix}\begin{matrix}{\pi_{t + 1} = {\pi_{t} + {\eta \frac{\partial I}{\partial\pi}}}} \\{{= {\pi_{t} + {\eta {\sum\limits_{i = 1}^{N}\; {\frac{\partial I}{\partial w_{i}}\frac{\partial w_{i}}{\partial\pi}}}}}},}\end{matrix} & (10)\end{matrix}$

where I(R, {right arrow over (W)}*) can be written as,

$\begin{matrix}{{I\left( {R,{\overset{\rightarrow}{W}}^{*}} \right)} = {\sum\limits_{r \in R}^{\;}\; {\int_{w^{*}}^{\;}{{p\left( {r,w^{*}} \right)}\log \frac{p\left( {r,w^{*}} \right)}{{p(r)}{p\left( w^{*} \right)}}\ {{w^{*}}.}}}}} & (11)\end{matrix}$

The provisioning manager 128 uses the data collected on the basedeployment to perform the search for the optimal projection. Since theprovisioning manager 128 uses workload and performance data collectedfrom the base deployment during the actual application runtime there isno additional cost in generating training data for the searching of theoptimal projection. In addition, as the cost relationship betweendifferent types of requests is independent of deployments, theprovisioning manager 128 applies the learned π to the training processof the reference model.

To determine the number of dimensions in the projected workload vector{right arrow over (W)}*, the user can choose the acceptable time lengthof automatic experiments and then use this information to derive thedimensions of {right arrow over (W)}*. For instance, suppose a userspecifies that the experiment of each deployment should not exceed 30minutes. If the performance measurement of a given workload can be donein 30 seconds the total number of workload compositions that can betested on one deployment is 60 (60×½=30). If a 10% random sampling ofworkload composition is good enough for model training and there are 5different levels for the request rate, there is a total population of600 (60/0.1=600) workload compositions which approximately correspondsto a dimension of 4 in {right arrow over (W)}* (5⁴=625≈600). Note thatthe user can also specify a high level cost requirement for modelbuilding, e.g., the maximum time for experiment or even the totalmonetary cost for experiment. Therefore, the dimension of {right arrowover (W)}* can be derived based on the above process, the number ofdeployments needed to test for collecting data and the virtual instancepricing policy.

With the prediction process discussed above, the provision plan selector210 of the provision manager 128 is able to identify and select theoptimal provisioning plan for an application. The provision planselector 210 explores all candidate provisioning plans and estimates thecost (monetary cost such as virtual machine renting fee which can beeasily computed based on the pricing policy of a cloud platform) andperformance (obtained by the prediction method discussed above) of eachcandidate plan. The optimal plan is the one with the lowest cost andperformance that satisfies the performance goal. As the cost estimationand performance prediction introduces trivial computational cost, theoverall search process can often be completed within a few seconds. Inaddition, the performance (prediction model, once trained) can berepeated used for different planning tasks with different performancegoals.

Operational Flow Diagrams

FIGS. 5-7 illustrate operational flow diagrams for various embodimentsof the present invention. The methods depicted in FIGS. 5-7 may beembodied in a computer-readable medium containing computer-readable codesuch that a series of steps are performed when the computer-readablecode is executed on a computing device. In some implementations, certainsteps of the methods may be combined, performed simultaneously or in adifferent order, or perhaps omitted, without deviating from the spiritand scope of the embodiments. Thus, while the method steps are describedand illustrated in a particular sequence, use of a specific sequence ofsteps is not meant to imply any limitations on the invention. Changesmay be made with regards to the sequence of steps without departing fromthe spirit or scope of the present invention. Use of a particularsequence is therefore, not to be taken in a limiting sense, and thescope of the present invention is defined only by the appended claims.

FIG. 5 is an operational flow diagram illustrating one example of anoverall process for performing prediction-based provisioning planning ina cloud computing environment. The operational flow diagram of FIG. 5begins at step 502 and flows directly to step 504. The provision manager128, at step 504, receives a user's request to deploy a multi-tier cloudapplication in the cloud computing environment 104. This requestincludes a specification of the application to be deployed, the expectedworkload range, and the expected performance. The provision manager 128,at step 506, deploys the application in an over-provisioned setting.While the application is running in the cloud infrastructure, theprovision manager 128, at step 508, monitors the workloads andperformance of the application, and stores the corresponding monitoringdata.

The provision manager 128, at step 510, trains a cross-tier performancemodel 212 based on the collected workloads and performance data. Theprovision manager 128, at step 512, replicates the application andperforms one or more automatic experiments to learn the performancecharacteristics of different deployment options. The provision manager128, at step 514, monitors and records the performance changes caused bythe deployment changes. The provision manager 128, at step 516, trainsper-tier performance model 214 based on the workloads and performancedata collected in the automatic experiments. The provision manager 128,at step 518, predicts the performance of all candidate plans. Theprovision manager 128, at step 520, identifies and selects a candidateprovisioning plan that meets the user specified performance goal and hasthe lowest virtual machine instance renting cost. The control flow thenexits at step 522.

FIG. 6 is an operational flow diagram illustrating one example of aprocess for predicting the performance of candidate plans. Theoperational flow diagram of FIG. 6 begins at step 602 and flows directlyto step 604. The provisioning manager 128, at step 604, uses thecross-tier model 212 to predict the base deployment performance. Theprovisioning manager 128, at step 606, selects a candidate provisioningplan. The provisioning manager 128, at step 608, uses the per-tierdifferential performance model 214 to predict the performance change ata given tier. The provisioning manager 128, at step 610, determines ifall tiers have been tested. If the result of this determination isnegative, the control flow returns to step 6008 and the prediction ifperformed for a new tier. If the result of this determination ispositive, the provisioning manager, at step 612, combines the predictedbase deployment performance and the predicted performance changes at alltiers. The provisioning manager 128, at step 614, determines if allcandidate plans have been explored. If the result of this determinationis negative, the control flow returns to step 606. If the result of thisdetermination is positive, the provisioning manager 128, at step 616,outputs the candidate plan that meets the performance goal specified bythe user with the lowest cost. The control flow exits at step 618.

FIG. 7 is an operational flow diagram illustrating one example of aprocess for performing automatic provisioning experiments on a currentlydeployed cloud application. The operational flow diagram of FIG. 7begins at step 702 and flows directly to step 704. The provisioningmanager 128, at step 704, replicates the deployed application. Theprovisioning manager 128, at step 706, selects a tier of the deployment.The provisioning manager 128, at step 708, changes the type of VM withinthe current tier. The provisioning manager 128, at step 710, tests thechanged deployment with different workloads and measures the performancechanges.

The provisioning manager 128, at step 712, changes the number of VMswithin the current tier. The provisioning manager 128, at step 714,tests the changed deployment with different workloads and measures theperformance changes. The provisioning manager 128, at step 716,determines if all sizes of VMs (all number of VMs) have been tested. Ifthe result of this determination is negative, the control flow returnsto step 712. If the result of this determination is positive, theprovisioning manager 128, at step 718, determines of all VM types havebeen tested. If the result of this determination is negative, thecontrol flow returns to step 708. If the result of this determination ispositive, the provisioning manager 128, at step 720, determines if alltiers within the deployment have been tested. If the result of thisdetermination is negative, the control flow returns to step 704. If theresult of this determination is positive, the control flow exits at step722.

Cloud Computing

It should be understood that although the following includes a detaileddiscussion on cloud computing, implementation of the teachings recitedherein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed, including client-server and peer-to-peer computingenvironments. For example, various embodiments of the present inventionare applicable to any computing environment with a virtualizedinfrastructure or any other type of computing environment.

For convenience, this discussion includes the following definitionswhich have been derived from the “Draft NIST Working Definition of CloudComputing” by Peter Mell and Tim Grance, dated Oct. 7, 2009, which iscited in an IDS filed herewith, and a copy of which is attached thereto.However, it should be noted that cloud computing environments that areapplicable to one or more embodiments of the present invention are notrequired to correspond to the following definitions and characteristicsgiven below or in the “Draft NIST Working Definition of Cloud Computing”publication. It should also be noted that the following definitions,characteristics, and discussions of cloud computing are given asnon-limiting examples.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. A cloud model may include atleast five characteristics, at least three service models, and at leastfour deployment models.

Cloud characteristics may include: on-demand self-service; broad networkaccess; resource pooling; rapid elasticity; and measured service. Cloudservice models may include: software as a service (SaaS); platform as aservice (PaaS); and infrastructure as a service (IaaS). Cloud deploymentmodels may include: private cloud; community cloud; public cloud; andhybrid cloud.

With on-demand self-service a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with a serviceprovider. With broad network access capabilities are available over anetwork and accessed through standard mechanisms that promote use byheterogeneous thin or thick client platforms (e.g., mobile phones,laptops, and personal digital assistants (PDAs)). With resource poolingcomputing resources of a provider are pooled to serve multiple consumersusing a multi-tenant model, with different physical and virtualresources dynamically assigned and reassigned according to demand. Inresource pooling there is a sense of location independence in that theconsumer generally has no control or knowledge over the exact locationof the provided resources but may be able to specify location at ahigher level of abstraction (e.g., country, state, or datacenter).

With rapid elasticity capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale-out and berapidly released to quickly scale-in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time. With measured service cloudsystems automatically control and optimize resource use by leveraging ametering capability at some level of abstraction that is appropriate tothe type of service (e.g., storage, processing, bandwidth, and activeuser accounts). Resource usage can be monitored, controlled, andreported providing transparency for both the provider and consumer ofthe utilized service.

In an SaaS model the capability provided to the consumer is to useapplications of a provider that are running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail). Inthe SaaS model, the consumer does not manage or control the underlyingcloud infrastructure (including networks, servers, operating systems,storage, or even individual application capabilities), with the possibleexception of limited user-specific application configuration settings.

In a PaaS model a cloud consumer can deploy consumer-created or acquiredapplications (created using programming languages and tools supported bythe provider) onto the cloud infrastructure. In the PaaS model, theconsumer does not manage or control the underlying cloud infrastructure(including networks, servers, operating systems, or storage), but hascontrol over deployed applications and possibly application hostingenvironment configurations.

In an IaaS service model a cloud consumer can provision processing,storage, networks, and other fundamental computing resources where theconsumer is able to deploy and run arbitrary software (which can includeoperating systems and applications). In the IaaS model, the consumerdoes not manage or control the underlying cloud infrastructure but hascontrol over operating systems, storage, deployed applications, andpossibly limited control of select networking components (e.g., hostfirewalls).

In a private cloud deployment model the cloud infrastructure is operatedsolely for an organization. The cloud infrastructure may be managed bythe organization or a third party and may exist on-premises oroff-premises. In a community cloud deployment model the cloudinfrastructure is shared by several organizations and supports aspecific community that has shared concerns (e.g., mission, securityrequirements, policy, and compliance considerations). The cloudinfrastructure may be managed by the organizations or a third party andmay exist on-premises or off-premises. In a public cloud deploymentmodel the cloud infrastructure is made available to the general publicor a large industry group and is owned by an organization selling cloudservices.

In a hybrid cloud deployment model the cloud infrastructure is acomposition of two or more clouds (private, community, or public) thatremain unique entities but are bound together by standardized orproprietary technology that enables data and application portability(e.g., cloud bursting for load-balancing between clouds). In general, acloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure that includes anetwork of interconnected nodes.

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 8, a schematic of an example of a cloud computingnode is shown. Cloud computing node 800 is only one example of asuitable cloud computing node and is not intended to suggest anylimitation as to the scope of use or functionality of embodiments of theinvention described herein. Regardless, cloud computing node 800 iscapable of being implemented and/or performing any of the functionalityset forth hereinabove.

In cloud computing node 800 there is a computer system/server 802, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 802 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Computer system/server 802 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 802 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

As shown in FIG. 8, computer system/server 802 in cloud computing node800 is shown in the form of a general-purpose computing device. Thecomponents of computer system/server 802 may include, but are notlimited to, one or more processors or processing units 804, a systemmemory 806, and a bus 808 that couples various system componentsincluding system memory 806 to processor 804.

Bus 808 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus.

Computer system/server 802 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 1002, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 806, in one embodiment, comprises the provisioning manager128, the cross-tier models 212, and the per-tier models 214 discussedabove. The provisioning manager 128 can also be implemented in hardwareas well. The system memory 806 can include computer system readablemedia in the form of volatile memory, such as random access memory (RAM)810 and/or cache memory 812. Computer system/server 802 may furtherinclude other removable/non-removable, volatile/non-volatile computersystem storage media. By way of example only, storage system 814 can beprovided for reading from and writing to a non-removable, non-volatilemagnetic media (not shown and typically called a “hard drive”). Althoughnot shown, a magnetic disk drive for reading from and writing to aremovable, non-volatile magnetic disk (e.g., a “floppy disk”), and anoptical disk drive for reading from or writing to a removable,non-volatile optical disk such as a CD-ROM, DVD-ROM or other opticalmedia can be provided. In such instances, each can be connected to bus808 by one or more data media interfaces. As will be further depictedand described below, memory 806 may include at least one program producthaving a set (e.g., at least one) of program modules that are configuredto carry out the functions of various embodiments of the invention.

Program/utility 816, having a set (at least one) of program modules 818,may be stored in memory 806 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 818 generally carry out the functionsand/or methodologies of various embodiments of the invention asdescribed herein.

Computer system/server 802 may also communicate with one or moreexternal devices 1020 such as a keyboard, a pointing device, a display822, etc.; one or more devices that enable a user to interact withcomputer system/server 802; and/or any devices (e.g., network card,modem, etc.) that enable computer system/server 802 to communicate withone or more other computing devices. Such communication can occur viaI/O interfaces 824. Still yet, computer system/server 1002 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 826. As depicted, network adapter 826communicates with the other components of computer system/server 802 viabus 808. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 802. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

Referring now to FIG. 9, illustrative cloud computing environment 902 isdepicted. As shown, cloud computing environment 902 comprises one ormore cloud computing nodes 800 with which local computing devices usedby cloud consumers, such as, for example, personal digital assistant(PDA) or cellular telephone 904, desktop computer 906, laptop computer908, and/or automobile computer system 910 may communicate. Nodes 800may communicate with one another. They may be grouped (not shown)physically or virtually, in one or more networks, such as Private,Community, Public, or Hybrid clouds as described hereinabove, or acombination thereof. This allows cloud computing environment 902 tooffer infrastructure, platforms and/or software as services for which acloud consumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 904, 906,908, 910 shown in FIG. 9 are intended to be illustrative only and thatcomputing nodes 800 and cloud computing environment 902 can communicatewith any type of computerized device over any type of network and/ornetwork addressable connection (e.g., using a web browser).

Referring now to FIG. 10, a set of functional abstraction layersprovided by cloud computing environment 902 (FIG. 9) is shown. It shouldbe understood in advance that the components, layers, and functionsshown in FIG. 10 are intended to be illustrative only and embodiments ofthe invention are not limited thereto. As depicted, the following layersand corresponding functions are provided:

Hardware and software layer 1002 includes hardware and softwarecomponents. Examples of hardware components include mainframes, in oneexample IBM® zSeries® systems; RISC (Reduced Instruction Set Computer)architecture based servers, in one example IBM pSeries® systems; IBMxSeries® systems; IBM BladeCenter® systems; storage devices; networksand networking components. Examples of software components includenetwork application server software, in one example IBM WebSphere®application server software; and database software, in one example IBMDB2® database software. (IBM, zSeries, pSeries, xSeries, BladeCenter,WebSphere, and DB2 are trademarks of International Business MachinesCorporation registered in many jurisdictions worldwide)

Virtualization layer 1004 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers;virtual storage; virtual networks, including virtual private networks;virtual applications and operating systems; and virtual clients.

In one example, management layer 1006 may provide the functionsdescribed below. Resource provisioning provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricingprovide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal provides access to the cloud computing environment forconsumers and system administrators. Service level management providescloud computing resource allocation and management such that requiredservice levels are met. Service Level Agreement (SLA) planning andfulfillment provide pre-arrangement for, and procurement of, cloudcomputing resources for which a future requirement is anticipated inaccordance with an SLA.

Workloads layer 1008 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation; software development and lifecycle management; virtualclassroom education delivery; data analytics processing; transactionprocessing; and composable software bundle and virtual image assetdesign and creation.

Non-Limiting Examples

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method, or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit”,” “module”, or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention have been discussed above withreference to flowchart illustrations and/or block diagrams of methods,apparatus (systems), and computer program products according to variousembodiments of the invention. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The description of the present invention has been presented for purposesof illustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A computer program product for predictingperformance of a system comprising a plurality of server tiers, thecomputer program product comprising: a storage medium readable by aprocessing circuit and storing instructions for execution by theprocessing circuit for performing a method comprising: collecting, for abase allocation of computing resources across multiple server tiers inthe plurality of sever tiers, a first set of performance informationassociated with the multiple server tiers for a set of workloads;generating, on a tier-by-tier basis, a set of experimental allocationsof the computing resources, wherein each of the set of experimentalallocations varies the computing resources allocated by the baseallocation for a single server tier of the multiple server tiers;collecting, for each of the set of experimental allocations, a secondset of performance information associated with the single server tierfor a plurality of workloads; and predicting at least one performancecharacteristic of one or more candidate allocations of computingresources across the multiple server tiers for a given workload based onthe first set of performance information and the second set ofperformance information.
 2. The computer program product of claim 1,wherein the method further comprises: constructing a cross-tierperformance model associated with the multiple server tiers based on thefirst set of performance information, wherein the cross-tier performancemodel predicts at least one performance characteristic of the baseallocation for various workloads; and constructing a single-tierperformance model for each of the single server tiers associated withthe set of experimental allocations, wherein the single-tier performancemodel captures performance changes between the base allocation and thesingle server tier for each of the set of set of experimentalallocations associated with the single server tier.
 3. The computerprogram product of claim 2, wherein the performance changes are capturedusing non-parametric regression.
 4. The computer program product ofclaim 2, wherein predicting the at least one performance characteristicof the one or more candidate allocations comprises: determining at leastone performance characteristic of the base allocation for the givenworkload based on the cross-tier performance model; determining, for atleast one server tier of the multiple server tiers associated with thecandidate allocation, a delta between the performance characteristic ofthe base allocation and at least one performance characteristic of theserver tier for the given workload; and combining the performancecharacteristic of the base allocation with the delta to obtain thepredicted performance characteristic of the candidate allocation.
 5. Thecomputer program product of claim 4, wherein the method furthercomprises: predicting a workload at each server tier of the multipleserver tiers associated with the candidate allocation; identifying anactual workload from the given workload at each tier; and replacing thegiven workload with the actual workload that has been identified.
 6. Thecomputer program product of claim 5, wherein predicting the workload ateach server tier is based on at least one of actual workloads receivedby each server tier and workloads observed at each server tier.
 7. Thecomputer program product of claim 1, wherein the method furthercomprises: measuring performance data associated with the multipleserver tiers for plurality of different workload types; determining aset of correlations among the performance data across the differentworkload types; and reducing, based on the set of correlations, a numberof workloads required to collect at least one of the first performanceinformation and the second performance information.
 8. The computerprogram product of claim 7, wherein the set of correlations are capturedusing non-parametric regression.
 9. The computer program product ofclaim 1, wherein the method further comprises: identifying, based on thepredicting, a candidate allocation from the one or more candidateallocations that comprises a predicted performance characteristic thatsatisfies a performance goal specified by a user, and that is associatedwith a lowest monetary cost; and notifying the user of the identifiedcandidate allocation.
 10. The method of claim 1, wherein the multipleserver tiers comprise a web server tier, an application server tier, anda database server tier.
 11. The computer program product of claim 1,wherein the predicted performance characteristic is at least one of anaverage response time associated with a workload and a requestthroughput rate.
 12. The computer program product of claim 1, whereincomputing resources are varied within each of the set of experimentalallocations by varying at least one of a number of virtual machines anda type of virtual machines allocated for the single server tier.
 13. Thecomputer program product of claim 1, wherein the method furthercomprises: determining a cost relationship between different types ofrequests within a least one of the set of workloads and the plurality ofworkloads based on mutual information between a set of performanceassociated with one or more server tiers of the multiple server tiersand one or more workloads; and constructing groups of request types fromrequests within the one or more workloads based on the cost relationshipbetween different types of requests, wherein a cost of requests in asame group is defined in a same linear space.
 14. The computer programproduct of claim 1, wherein the method further comprises: reducing acomplexity of a performance prediction model used to predict the atleast one performance characteristic based on combining multiple requesttypes within one or more workloads into a given number of requestgroups, wherein the given number of request groups reduce at least onedimension of a workload vector used as an input to the performanceprediction model, and wherein the given number of request groups isdetermined by performing gradient ascent on mutual information between aset of performance associated with one or more server tiers of themultiple server tiers and one or more workloads.
 15. An informationprocessing system for predicting performance of a system comprising aplurality of server tiers, the information processing system comprising:a memory; a processor communicatively coupled to the memory; and aprovisioning manager communicatively coupled to the memory and theprocessor, wherein the provisioning manager is configured to perform amethod comprising: collecting, for a base allocation of computingresources across multiple server tiers in the plurality of sever tiers,a first set of performance information associated with the multipleserver tiers for a set of workloads; generating, on a tier-by-tierbasis, a set of experimental allocations of the computing resources,wherein each of the set of experimental allocations varies the computingresources allocated by the base allocation for a single server tier ofthe multiple server tiers; collecting, for each of the set ofexperimental allocations, a second set of performance informationassociated with the single server tier for a plurality of workloads; andpredicting at least one performance characteristic of one or morecandidate allocations of computing resources across the multiple servertiers for a given workload based on the first set of performanceinformation and the second set of performance information.
 16. Theinformation processing system of claim 15, wherein the method furthercomprises: constructing a cross-tier performance model associated withthe multiple server tiers based on the first set of performanceinformation, wherein the cross-tier performance model predicts at leastone performance characteristic of the base allocation for variousworkloads; and constructing a single-tier performance model for each ofthe single server tiers associated with the set of experimentalallocations, wherein the single-tier performance model capturesperformance changes between the base allocation and the single servertier for each of the set of set of experimental allocations associatedwith the single server tier.
 17. The information processing system ofclaim 16, wherein the performance changes are captured usingnon-parametric regression.
 18. The information processing system ofclaim 16, wherein predicting the at least one performance characteristicof the one or more candidate allocations comprises: determining at leastone performance characteristic of the base allocation for the givenworkload based on the cross-tier performance model; determining, for atleast one server tier of the multiple server tiers associated with thecandidate allocation, a delta between the performance characteristic ofthe base allocation and at least one performance characteristic of theserver tier for the given workload; and combining the performancecharacteristic of the base allocation with the delta to obtain thepredicted performance characteristic of the candidate allocation. 19.The information processing system of claim 18, wherein the methodfurther comprises: predicting a workload at each server tier of themultiple server tiers associated with the candidate allocation;identifying an actual workload from the given workload at each tier; andreplacing the given workload with the actual workload that has beenidentified.
 20. The information processing system of claim 15, whereinthe method further comprises: measuring performance data associated withthe multiple server tiers for plurality of different workload types;determining a set of correlations among the performance data across thedifferent workload types; and reducing, based on the set ofcorrelations, a number of workloads required to collect at least one ofthe first performance information and the second performanceinformation.
 21. The information processing system of claim 15, whereinthe method further comprises: identifying, based on the predicting, acandidate allocation from the one or more candidate allocations thatcomprises a predicted performance characteristic that satisfies aperformance goal specified by a user, and that is associated with alowest monetary cost; and notifying the user of the identified candidateallocation.
 22. The information processing system of claim 15, whereinthe predicted performance characteristic is at least one of an averageresponse time associated with a workload and a request throughput rate.23. The information processing system of claim 15, wherein computingresources are varied within each of the set of experimental allocationsby varying at least one of a number of virtual machines and a type ofvirtual machines allocated for the single server tier.
 24. Theinformation processing system of claim 15, wherein the method furthercomprises: determining a cost relationship between different types ofrequests within a least one of the set of workloads and the plurality ofworkloads based on mutual information between a set of performanceassociated with one or more server tiers of the multiple server tiersand one or more workloads; and constructing groups of request types fromrequests within the one or more workloads based on the cost relationshipbetween different types of requests, wherein a cost of requests in asame group is defined in a same linear space.
 25. The informationprocessing system of claim 15, wherein the method further comprises:reducing a complexity of a performance prediction model used to predictthe at least one performance characteristic based on combining multiplerequest types within one or more workloads into a given number ofrequest groups, wherein the given number of request groups reduce atleast one dimension of a workload vector used as an input to theperformance prediction model, and wherein the given number of requestgroups is determined by performing gradient ascent on mutual informationbetween a set of performance associated with one or more server tiers ofthe multiple server tiers and one or more workloads.